Immersive Training Environment Research Articles

Systematic review of motion capture in virtual reality: Enhancing the precision of sports training

In the modern era of sports training, the synergy between motion capture and Virtual Reality (VR) offers an innovative approach to enhancing training precision. This systematic review delves into the application of motion capture within VR for sports training, highlighting its transformative potential. Through a comprehensive literature search, we examined the myriad applications, from physical conditioning enhancements to accelerated rehabilitation processes. Our findings underscore the capability of real-time feedback, immersive training environments, and tailored regimes that this fusion provides. However, despite its promise, challenges such as hardware constraints, data processing complexities, and interaction interface limitations persist. Future trajectories indicate an increasing influence of AI and deep learning, promising more sophisticated hardware and a broader spectrum of applications, including niche sports disciplines. The review concludes with an emphasis on the wider societal implications, suggesting a shift towards a holistic athlete well-being approach.

Application of Artificial Intelligence in Paramedic Education: Current Scenario and Future Perspective: A Narrative Review

Background: Artificial intelligence (AI) has the potential to revolutionise paramedic education. As well as allowing for personalised learning experiences tailored to individual needs and learning styles, it can provide simulations, intelligent tutoring systems, automated grading and assessment, and predictive analytics. Objective: To investigate the role of artificial intelligence in transforming the landscape of paramedic education and evaluate its potential to improve learning outcomes. Methods: This review presented the role of AI in paramedic education and its perspective over the past twenty years. It included high-quality data and comprehensive investigations of articles available in renowned databases. Results: AI-based training and simulation technologies, such as virtual patients, surgical simulators, and intelligent tutoring systems,are increasingly being used in paramedic education. Virtual patients use computer-generated avatars to display symptoms and react to therapies, while surgical simulators use accurate anatomical models and haptic feedback devices to simulate surgical operations. Conclusion: AI has the potential to fundamentally alter how students learn, the kind of education they receive, and the efficiency with which healthcare is delivered. It can create immersive training environments, analyse medical data, and help students feel more competent, confident, and capable. This potential can be harnessed to enhance paramedic education and improve patient care outcomes.

Technological Breakthroughs in Sport: Current Practice and Future Potential of Artificial Intelligence, Virtual Reality, Augmented Reality, and Modern Data Visualization in Performance Analysis

We are currently witnessing an unprecedented era of digital transformation in sports, driven by the revolutions in Artificial Intelligence (AI), Virtual Reality (VR), Augmented Reality (AR), and Data Visualization (DV). These technologies hold the promise of redefining sports performance analysis, automating data collection, creating immersive training environments, and enhancing decision-making processes. Traditionally, performance analysis in sports relied on manual data collection, subjective observations, and standard statistical models. These methods, while effective, had limitations in terms of time and subjectivity. However, recent advances in technology have ushered in a new era of objective and real-time performance analysis. AI has revolutionized sports analysis by streamlining data collection, processing vast datasets, and automating information synthesis. VR introduces highly realistic training environments, allowing athletes to train and refine their skills in controlled settings. AR overlays digital information onto the real sports environment, providing real-time feedback and facilitating tactical planning. DV techniques convert complex data into visual representations, improving the understanding of performance metrics. In this paper, we explore the potential of these emerging technologies to transform sports performance analysis, offering valuable resources to coaches and athletes. We aim to enhance athletes’ performance, optimize training strategies, and inform decision-making processes. Additionally, we identify challenges and propose solutions for integrating these technologies into current sports analysis practices. This narrative review provides a comprehensive analysis of the historical context and evolution of performance analysis in sports science, highlighting current methods’ merits and limitations. It delves into the transformative potential of AI, VR, AR, and DV, offering insights into how these tools can be integrated into a theoretical model.

Evaluation of a Virtual Reality Platform to Train Stress Management Skills for a Defense Workforce: Multisite, Mixed Methods Feasibility Study.

Psychological stress-related injuries within first-responder organizations have created a need for the implementation of effective stress management training. Most stress management training solutions have limitations associated with scaled adoption within the workforce. For instance, those that are effective in civilian populations often do not align with the human performance culture embedded within first-responder organizations. Programs involving expert-led instructions that are high in quality are often expensive. This study sought to evaluate a tailored stress management training platform within the existing training schedule of the Australian Defense Force (ADF). The platform, known as Performance Edge (PE), is a novel virtual reality (VR) and biofeedback-enabled stress management skills training platform. Focusing on practical training of well-established skills and strategies, the platform was designed to take advantage of VR technology to generate an immersive and private training environment. This study aimed to assess the feasibility of delivering the VR platform within the existing group-based training context and intended training population. In this setting, the study further aimed to collect data on critical predictors of user acceptance and technology adoption in education, including perceived usability, usefulness, and engagement, while also assessing training impacts. This study used a mixed methods, multisite approach to collect observational, self-reported, and biometric data from both training staff and trainers within a real-world "on-base" training context in the ADF. Validated scales include the Presence Questionnaire and User Engagement Scale for perceived usefulness, usability, and engagement, as well as the State Mindfulness Scale and Relaxation Inventory, to gain insights into immediate training impacts for specific training modules. Additional surveys were specifically developed to assess implementation feedback, intention to use skills, and perceived training impact and value. PE training was delivered to 189 ADF trainees over 372 training sessions. The platform was easy to use at an individual level and was feasible to deliver in a classroom setting. Trainee feedback consistently showed high levels of engagement and a sense of presence with the training content and environment. PE is overall perceived as an effective and useful training tool. Self-report and objective indices confirmed knowledge improvement, increased skill confidence, and increased competency after training. Specific training elements resulted in increased state mindfulness, increased physical relaxation, and reduced breathing rate. The ability to practice cognitive strategies in a diverse, private, and immersive training environment while in a group setting was highlighted as particularly valuable. This study found the VR-based platform (PE) to be a feasible stress management training solution for group-based training delivery in a defense population. Furthermore, the intended end users, both trainers and trainees, perceive the platform to be usable, useful, engaging, and effective for training, suggesting end-user acceptance and potential for technology adoption.

Research and design of key technology and system architecture on simulation confrontation training

Aiming at the problems of unreal training scenes, unintelligent training agents and difficult data interaction in the process of simulation confrontation training, this paper combines image processing, neural network, GBB(general blackboard) technologies with system simulation technology, research around realistic complex environment modeling technology, confrontation agents modeling technology, and multi-model data interaction technology, then puts forward practical solutions. On this basis, a simulation confrontation training system integrating training management, scene development and resource management is designed, which provides a feasible idea and reference for the construction of an immersive, high-intensity and agile interactive confrontation training environment.

Design and feasibility study of a HCPS framework-based VR alpine skiing decision-making training system

Alpine skiing beginners are easily injured if they fail make the right decisions when faced with unexpected situations. Excellent decision-making requires extensive training, while alpine skiing training is limited by time and space. To address these issues, this paper combines VR and machine learning techniques to develop a high fidelity VR skiing decision-making training system based on HCPS framework. The aim is to train beginner skiers in decision-making by creating automated and repeatable virtual scenarios. Firstly, physical world (ski resort environment, and user motions) was mapped into the VR world using the HCPS framework. Afterwards, the user’s decision motions were made based on the scenarios provided, while being identified through an improved SVM (94.5% accuracy). The motion recognition results were fed into the expert system to assess the feasibility of the decision-making. Finally, the feasibility of the system was assessed with a combination of subjective and objective approaches. The results showed that the VR system significantly improved the participants’ skiing decision-making ability and speed compared to traditional 2D video training. The high fidelity environment reduced participants’ motion sickness symptoms and resulted in a better sensory experience. In addition, participants were more focused in the high fidelity environment. Overall, these results demonstrated the construct validity of the virtual ski decision-making training system. This study changes the traditional training method to create an immersive training environment, and provides an effective avenue for the development of sports training.

Hospital-Based, Community Teaching Kitchen Integrates Diabetes Education, Culinary Medicine, and Food Assistance: Case Study During the COVID-19 Pandemic.

Recent USDA Economic Research Service Population Survey cites a stabilization of food insecurity overall in the USA between 2019 and 2020, but Black, Hispanic, and all households with children cited increases - underscoring that the COVID-19 pandemic caused severe disruptions to food insecurity for historically disenfranchised populations. Describe lessons learned, considerations, and recommendations from the experience of a community teaching kitchen (CTK) in addressing food insecurity and chronic disease management among patients during the COVID-19 pandemic. The Providence CTK is co-located at Providence Milwaukie Hospital in Portland, Oregon. Providence CTK serves patients who report a higher prevalence of food insecurity and multiple chronic conditions. Providence CTK has five components: chronic disease self-management education, culinary nutrition education, patient navigation, a medical referral-based food pantry (Family Market), and an immersive training environment. CTK staff highlight that they provided food and education support when it was needed most, leveraged existing partnerships and staffing to sustain operations and Family Market accessibility, shifted delivery of educational services based-on billing and virtual service considerations, and repurposed roles to support evolving needs. The Providence CTK case study provides a blueprint for how healthcare organizations could design a model of culinary nutrition education that is immersive, empowering, and inclusive.

Development of machine tool operation simulator by VR

In the manufacturing field, various types of machine tools are widely spread and used. However, machine tools have mechanisms that rotate at high speed, generate heat, large forces which can be dangerous for inexperienced users. Therefore, safe training system is required. Virtual reality (VR) is inexpensive compared to machine tools and can provide an immersive training environment for a large number of trainee. In this study, we developed a training system for machining in a virtual space for a desktop machine tool. Furthermore, in order to realize the digital twin, the behavior of the actual machine and that of the virtual machine can be synchronized using TCP/IP, and the behavior of the table of the actual machine can be monitored.

Feasibility Study on Using AI and VR for Decision-Making Training of Basketball Players

Decision-making plays an essential role in basketball offense. Offensive players must make effective decisions to score a basket in various defensive situations. Virtual reality (VR) has been widely used in the training of athletes to strengthen their ability to make optimal decisions by creating controllable repeatable training scenarios. In this article, an action-aware offensive decision-making training system for basketball using VR and artificial intelligence is proposed. The proposed system is composed of different virtual defensive scenarios and an offensive action recognition framework. Trainees wearing head-mounted display and a motion capture suit are trained by intuitively interacting with the VR system and receive decision suggestions when a bad one is made. This study changes the training media and methods to create an immersive training environment during the training phase and evaluates the training effectiveness. These training scenarios are a conventional tactics board, the proposed VR system with a prerecorded <inline-formula><tex-math notation="LaTeX">$360^\circ$</tex-math></inline-formula> panorama video, and the VR system with computer-simulated virtual content. Results indicate that the training scenario affects the training in terms of decision time.

An Adaptive Game-Based Learning Strategy for Children Road Safety Education and Practice in Virtual Space.

Virtual reality (VR) has been widely used as a tool to assist people by letting them learn and simulate situations that are too dangerous and risky to practice in real life, and one of these is road safety training for children. Traditional video- and presentation-based road safety training has average output results as it lacks physical practice and the involvement of children during training, without any practical testing examination to check the learned abilities of a child before their exposure to real-world environments. Therefore, in this paper, we propose a 3D realistic open-ended VR and Kinect sensor-based training setup using the Unity game engine, wherein children are educated and involved in road safety exercises. The proposed system applies the concepts of VR in a game-like setting to let the children learn about traffic rules and practice them in their homes without any risk of being exposed to the outside environment. Thus, with our interactive and immersive training environment, we aim to minimize road accidents involving children and contribute to the generic domain of healthcare. Furthermore, the proposed framework evaluates the overall performance of the students in a virtual environment (VE) to develop their road-awareness skills. To ensure safety, the proposed system has an extra examination layer for children’s abilities evaluation, whereby a child is considered fit for real-world practice in cases where they fulfil certain criteria by achieving set scores. To show the robustness and stability of the proposed system, we conduct four types of subjective activities by involving a group of ten students with average grades in their classes. The experimental results show the positive effect of the proposed system in improving the road crossing behavior of the children.

Preliminary study on mechanical characteristics of maxillofacial soft and hard tissues for virtual surgery

PurposeVirtual surgery system can provide us a realistic and immersive training environment, in which haptic force-feedback gives operators ‘touching feeling.’ Appropriate deformation models of soft and hard tissues are required for the achievement of real-time haptic feedback. To improve accuracy of modeling and haptic feedback simulation for maxillofacial virtual surgery, mechanical characteristics of soft and hard tissues should be explored.MethodsCraniofacial soft tissues from one male and female cadavers were divided into two layers: skin and muscle. Maxillofacial tissues were divided into frontal, chin, temporalis, masseter regions. Insertion and cutting process were conducted using VMX42 5-axis linkage system and recorded by piezoelectric dynamometer. Maximum stiffness values were analyzed, and insertion curves before puncture were fitted using a polynomial model. Elasticity modulus and hardness of maxillofacial hard tissues were measured and analyzed using Berkovich nanoindentation.ResultsTissues in different maxillofacial regions, as well as from different layers (skin and muscle), displayed various mechanical performance. Maximum stiffness values and cutting force of soft tissues in male and female had significant difference. The third-order polynomial was demonstrated to fit the insertion curves well before puncture. Furthermore, elasticity modulus and hardness of enamel were significantly greater than that of zygoma, maxilla and mandible.ConclusionMechanical properties of hard tissues are relatively stable, which can be applied in virtual surgery system for physical model construction. Insertion model and cutting force for soft tissues are meaningful and applicable and can be utilized to promote the accuracy of response for haptic feedback sensations.

Virtual Operating Room Simulation Setup (VORSS) for Procedural Training in Minimally Invasive Surgery \u2013 a Pilot Study

Virtual reality (VR) training is widely used in several minimal invasive surgery (MIS) training curricula for procedural training. However, VR training in its current state lack immersive training environments, such as using head-mounted displays that is implemented in military or aviation training and even entertainment. The virtual operating room simulation setup (VORSS) is explored in this study to determine the effectiveness of immersive training in MIS. Twenty-eight surgeons and surgical trainees performed a laparoscopic cholecystectomy on the VORSS comprising of a head-mounted 360-degree realistic OR surrounding on a VR laparoscopic simulator. The VORSS replicated a full setup of instruments and surgical team-members as well as some of the distractions occurring during surgical procedures. Questionnaires were followed by semi-structured interviews to collect the data. Experts and novices found the VORSS to be intuitive and easy to use (p = 0.001). The outcome of the usability test, applying QUESI and NASA-TLX, reflected the usability of the VORSS (p < 0.05), at the cognitive level, which indicates a good sense of immersion and satisfaction, when performing the procedure within VORSS. The need for personalized experience within the setup was strongly noted from most of the participants. The VORSS for procedural training has the potential to become a useful tool to provide immersive training in MIS surgery. Further optimizing of the VORSS realism and introduction of distractors in the OR should result in an improvement of the system.

User perspective and higher cognitive task-loads influence movement and performance in immersive training environments

BackgroundIn virtual reality (VR) applications such as games, virtual training, and interactive neurorehabilitation, one can employ either the first-person user perspective or the third-person perspective to perceive the virtual environment; however, applications rarely offer both perspectives for the same task. We used a targeted-reaching task in a large-scale virtual reality environment (N=30 healthy volunteers) to evaluate the effects of user perspective on the head and upper extremity movements, and on user performance. We further evaluated how different cognitive challenges would modulate these effects. Finally, we obtained the user-reported engagement level under the different perspectives.ResultsWe found that first-person perspective resulted in larger head movements (3.52±1.3m) than the third-person perspective (2.41±0.7m). First-person perspective also resulted in more upper-extremity movement (30.08±7.28m compared to 26.66±4.86m) and longer completion times (61.3±16.4s compared to 53±10.4s) for more challenging tasks such as the “flipped mode”, in which moving one arm causes the opposite virtual arm to move. We observed no significant effect of user perspective alone on the success rate. Subjects reported experiencing roughly the same level of engagement in both first-person and third-person perspectives (F(1.58)=0.9,P=.445).ConclusionUser perspective and its interaction with higher-cognitive load tasks influences the extent of movement and user performance in a virtual theater environment, and may influence the choice of the interface type (first or third person) in immersive training depending on the user conditions and exercise requirements.

Situational awareness measurement in a simulation-based training framework for offshore well control operations

Human factors are identified as the major contributor to oil and gas drilling and operations related accidents. One major limitation of the currently available simulation-based training programs is the lack of objective and validated measures for human factors and non-technical skills. These objective measures are necessary to evaluate the skills and training of drilling crews. Situational awareness (SA) describes the person's perception, comprehension, and prediction in dynamic environments. SA is critical in time-sensitive decision making, which may be especially pertinent for kick-detection and blowout mitigation. This paper presents the analyses of multiple objective SA measures in an immersive training environment. The SA for a sample of engineering-in-training students (n = 13) was measured using a well control incident simulator. The data analyses include knowledge tests, a voice log analysis, usability measures, check-lists compliance, pre/post performance surveys, and eye-tracking data. The results show that task knowledge (e.g. current pressure readings, current bit depth) was predictive of overall blowout mitigation performance. Further, the usability results provide some preliminary insights into task risk perceptions, task difficulty, and perceptions of communication. The results may help design an objective simulation-based training framework for human factors and non-technical skills measurement in oil and gas well control operations.

Real-Time Needle Force Modeling for VR-Based Renal Biopsy Training with Respiratory Motion Using Direct Clinical Data.

Realistic tool-tissue interactive modeling has been recognized as an essential requirement in the training of virtual surgery. A virtual basic surgical training framework integrated with real-time force rendering has been recognized as one of the most immersive implementations in medical education. Yet, compared to the original intraoperative data, there has always been an argument that these data are represented by lower fidelity in virtual surgical training. In this paper, a dynamic biomechanics experimental framework is designed to achieve a highly immersive haptic sensation during the biopsy therapy with human respiratory motion; it is the first time to introduce the idea of periodic extension idea into the dynamic percutaneous force modeling. Clinical evaluation is conducted and performed in the Yunnan First People's Hospital, which not only demonstrated a higher fitting degree (AVG: 99.36%) with the intraoperation data than previous algorithms (AVG: 87.83%, 72.07%, and 66.70%) but also shows a universal fitting range with multilayer tissue. 27 urologists comprising 18 novices and 9 professors were invited to the VR-based training evaluation based on the proposed haptic rendering solution. Subjective and objective results demonstrated higher performance than the existing benchmark training simulator. Combining these in a systematic approach, tuned with specific fidelity requirements, haptically enabled medical simulation systems would be able to provide a more immersive and effective training environment.

How to Build a Patient-Specific Hybrid Simulator for Orthopaedic Open Surgery: Benefits and Limits of Mixed-Reality Using the Microsoft HoloLens.

Orthopaedic simulators are popular in innovative surgical training programs, where trainees gain procedural experience in a safe and controlled environment. Recent studies suggest that an ideal simulator should combine haptic, visual, and audio technology to create an immersive training environment. This article explores the potentialities of mixed-reality using the HoloLens to develop a hybrid training system for orthopaedic open surgery. Hip arthroplasty, one of the most common orthopaedic procedures, was chosen as a benchmark to evaluate the proposed system. Patient-specific anatomical 3D models were extracted from a patient computed tomography to implement the virtual content and to fabricate the physical components of the simulator. Rapid prototyping was used to create synthetic bones. The Vuforia SDK was utilized to register virtual and physical contents. The Unity3D game engine was employed to develop the software allowing interactions with the virtual content using head movements, gestures, and voice commands. Quantitative tests were performed to estimate the accuracy of the system by evaluating the perceived position of augmented reality targets. Mean and maximum errors matched the requirements of the target application. Qualitative tests were carried out to evaluate workload and usability of the HoloLens for our orthopaedic simulator, considering visual and audio perception and interaction and ergonomics issues. The perceived overall workload was low, and the self-assessed performance was considered satisfactory. Visual and audio perception and gesture and voice interactions obtained a positive feedback. Postural discomfort and visual fatigue obtained a nonnegative evaluation for a simulation session of 40 minutes. These results encourage using mixed-reality to implement a hybrid simulator for orthopaedic open surgery. An optimal design of the simulation tasks and equipment setup is required to minimize the user discomfort. Future works will include Face Validity, Content Validity, and Construct Validity to complete the assessment of the hip arthroplasty simulator.

Personalized planning and training system for brachytherapy based on virtual reality

This paper presents a virtual planning and training system for brachytherapy based on virtual reality (VR). The purpose of this system is to facilitate preplanning and help doctors to more quickly and easily improve their skills in brachytherapy surgery. For unskilled doctors, this system can increase confidence and support the successful completion of surgery. The system is based on a VR system that can deliver a fully immersive training environment and involves three functions: simulation of the relocation template, arrangement of the puncture needles and the implantation of the seeds. We used Student’s t test to verify the efficiency of the system. Participants were required to complete both the training and a questionnaire to supply feedback. The participants subsequently performed simulation surgery on a dummy and reviewed their cumulative errors compared with the ideal condition. The results demonstrate that this VR system is able to effectively train unskilled doctors, especially young doctors. This system is appreciated by a variety of people, especially inexperienced and younger users, as it is easy to use and provides an enjoyable learning experience.

Interventional Radiology Training Using a Dynamic Medical Immersive Training Environment (DynaMITE)

Interventional Radiology Training Using a Dynamic Medical Immersive Training Environment (DynaMITE)

A High-Immersive Medical Training Platform Using Direct Intraoperative Data

The virtual training of primitive surgical procedures has been widely recognized as immersive and effective to medical education. Virtual basic surgical training framework integrated with multi-sensations rendering has been recognized as one of the most immersive implementations in medical education. Yet, compared with the original intraoperative data, there has always been an argument on the lower fidelity these data are represented in virtual surgical training. In this paper, a solution is proposed to achieve better training immersion by incorporating multiple higher-fidelity factors toward a trainee’s sensations (vision, touch, and hearing) during virtual training sessions. This was based on the proposal of a three-tier model to classify reasons leading to fidelity issues. This include: haptic factors, such as high-quality fitting of force models based on surgical data acquisition, the use of actual surgical instrument linked to desktop haptic devices; visual factors, such as patient-specific CT images segmentation and reconstruction from the original medical data; and hearing factors, such as variations of the sound of monitoring systems in the theatre under different surgical conditions. Twenty seven urologists comprising 18 novices and 9 professors were invited to test a virtual training system based on the proposed solution. Post-test values from both professors’ and novices’ groups demonstrated obvious improvements in comparison with pre-test values under both the subjective and objective criteria, the fitting rate of the whole puncture processing is 99.93%. Both the subjective and objective results demonstrated a higher performance than the existing benchmark training platform. Combining these in a systematic approach, tuned with specific fidelity requirements, haptically enabled training simulation systems would be able to provide a more immersive and effective training environment.

A novel approach to automatic detection of presentation slides in educational videos

Recent advancement in learning and teaching methodology experimented with virtual reality (VR)-based presentation form to create immersive learning and training environment. The quality of such educational VR applications not only relies on the virtual model, but the 2D presentation materials such as text, diagrams and figures. However, manual designing or seeking these educational resources is both labor intensive and time-consuming. In this paper, we introduce a new automatic algorithm to detect and extract presentation slides in educational videos, which will provide abundant resources for creating slide-based immersive presentation environment. The proposed approach mainly involves five core components: shot boundary detection, training instances collection, shot classification, slide region detection and slide transition detection. We conducted comparison experiment to evaluate the performance of the proposed method. The results indicate that, in comparison with peer method, the proposed method improves the precision of slide detection from 81.6 to 92.6% and recall from 74.7 to 86.3% on average. With the detected slides, content analyzer can be employed to further extract reusable elements, which can be used for developing VR-based educational applications.